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Why  is  the  COBP  Important 
to  Decisionmakers? 


•  The  Code  can  provide  insight  into  the 
assessment  process. 

•  The  Code  represents  a  standard  against 
which  assessments  can  be  compared. 

•  The  Code  helps  focus  interaction  with  the 
technical  community. 

-  Right  questions 

-  Right  resources 

-  Right  timing 
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Senior  Analysts  and  Leaders 
Recognize  these  Principles 

ORGANIZATION 

•  Interaction  between  Experts  and  the  Customer 
[V.  Roske,  Phalanx  1998] 

-  “What  is  the  question?”  and  then  “What  is  the  REAL 
question?” 

-  “Know  your  Customer” 

•  Decisionmaker 

•  Sponsor 

•  Bill  payer 

•  Stakeholder 

-  Collaboration 

•  Design  and  execution  of  assessments/campaigns 

•  Development  and  selection  of  models 
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Who  are  Decisionmakers? 

Decisionmaking  is  Often  Collaborative 

ORGANIZATION 

People  have  Different  Responsibilities: 

•  Decisionmaker 

-  Identify  the  problem  and  the  REAL  problem 

-  Inform  the  assessment  team  of  changes  in  the  problem 

-  Engage  with  analysts  in  the  cyclic  process  of  model  creation  and  adaptation 

•  Sponsor 

-  Terms  of  reference 

-  Access 

-  Product  /  Goal  definition 

•  Bill  Payer 

-  Assemble  resources 

-  Provide  timely  support 

•  Stakeholder 

-  Provide  perspective  and  feedback 

•  Perspectives  depend  on: 

-  Responsibility  /  Authority 

-  Decision  timeline  , 
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General  Questions  Implied  by  the  Code 

ORGANIZATION 

Problem  Formulation  and  Solution  Strategies: 

-  What  are  our  goals?  (What  are  we  trying  to  fix  or  change?) 

-  What  do  we  already  know  about  the  problem? 

-  What  resources  do  we  need  to  do  this  assessment? 

-  How  will  we  know  we’re  doing  better?  (metrics) 

Conduct  of  the  Study: 

-  What  role  should  the  Decisionmaker  play? 

-  What  are  the  unanticipated  challenges? 

Study  Products: 

-  What  did  we  learn? 

-  What  was  surprising  (and  why)? 

-  What  does  this  imply  for  (acquisition,  policy,  future  assessments)? 

-  Explicit  answers  to  the  REAL  question 

•  Supporting  evidence 

•  Clear  presentation 
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Participant  Roles  and  Relationships 


ORGANIZATION 
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Keys  for  Successful  Assessment 


Decisionmaker  Involvement  is  Essential 


•  Decisionmaker  involvement  needs  to: 

-  Be  Continuous 

-  Focus  on  Milestones 

•  Problem  Formulation  /  Solution  Strategies 

•  MoM,  Scenarios,  Human  and  Organizational  Issues 

•  Assess  Execution 

•  Products 

-  Ensure  Peer  Review 

•  Continuous 

•  On  Milestones 
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C2  Assessment  Process 


Prepare  for  Success 

-  Assessment  Team 

-  Other  Participants 


0 

External  Review 

o 

Product 

a 

Process 
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Assessment  Team’s  Responsibilities 


It  is  the  Assessment  Team’s  responsibility  to: 

•  Ensure  science  comes  before  engineering 

•  Drill  down  to  the  ‘REAL  Question’ 

•  Support  the  Decisionmaker's  need  for 
understanding 

•  Identify  peer  reviewers 
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Assessment  Teams  are  Multidisciplinary 


Military  Operators 
and  Subject  Matter/ 
Experts 


Mission  & 
C2 

Domain 

Expertise 


Applied  Sciences} 
Engineering  & 
Operations 
Research 


Scientific  Expertise 


Mathematicians 
Social  Scientists 
Physical  Scientists 
Biological  Scientists 


Technologists 
Software  Engineers 
Hardware  Engineers 
Systems  Engineers 
Data  Modelers 
Statisticians 
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Metrics 


•  Metrics  are  the  link 
between  goals  and  the 
assessment 

•  Decisionmakers  should 
be  involved  in 
developing  metrics 

•  The  model  should  fit 
the  goals  and  metrics, 
not  the  reverse 


Environment 


MoPE  -  focuses  on  policy  and  societal  outcomes 
MoFE  -  focuses  on  how  a  force  performs  its  mission 
MoCE  -  focuses  on  the  impact  of  C2  systems  within 
the  operational  context 
MoP  -  focuses  on  internal  system  structure, 
characteristics  and  behavior 
DP  -  focuses  on  the  properties  or  characteristics 
inherent  in  the  physical  C2  systems 
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C2  Assessment 

ORGANIZATION 

Message  to  Decisionmakers: 

•  Analysis  is  Not  Advocacy 

•  The  Decisionmaker  must  create  a  culture  where  this 
attitude  is  possible 

•  “What  is  the  REAL  Question?” 

•  Stay  involved 

•  Ensure  peer  review 

•  Ask  follow-up  questions 

“What  is  the  assessment  telling  us? 

“What  is  the  assessment  not  telling  us?” 
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